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VIII. Claims 

Appendix A Claims 
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Appendix C Related Proceedings 

I. REAL PARTY IN INTEREST 

The real party in interest for this appeal is: 

Mitsubishi Denki Kabushiki Kaisha 

II. RELATED APPEALS AND INTERFERENCES 

There are no other appeals, interferences, or judicial proceedings which will directly 
affect or be directly affected by or have a bearing on the Board's decision in this appeal. 

III. STATUS OF CLAIMS 

A. Total Number of Claims in Application 
There are 14 claims pending in application. 

B. Current Status of Claims 

1. Claims canceled: 1-5, and 11-32 

2. Claims withdrawn from consideration but not canceled: None 

3. Claims pending: 6-10 and 33-41 

4. Claims allowed: None 

5. Claims rejected: 6-10 and 33-41 

C. Claims On Appeal 

The claims on appeal are claims 6-10 and 33-41 

IV. STATUS OF AMENDMENTS 

Applicant filed an Amendment on January 9, 2009, amending claims 6 and 7, canceling 
claims 10, 33-35 and 38-40, and adding new claim 42. Applicant believed the amended to have 
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been entered, but in view of the Notification of Non-Compliant Appeal Brief received June 9, 
2009 and the discussion with Examiner To, Applicant understands that the January 9, 2009 
amendment was not entered. 

V. SUMMARY OF CLAIMED SUBJECT MATTER 

The present invention relates to searching for an image or sequence of images (still or 
video image) containing a two-dimensional projection of a specific three-dimensional object of 
interest to the searcher. This is done by searching through stored representations of images or 
sequences of images. The stored representations represent each image or sequence of images 
using object descriptors for three-dimensional objects in the image or sequence of images. Each 
object descriptor includes a plurality of perspective view descriptors (associated together to form 
an object descriptor) which represent different two-dimensional perspective views of the three- 
dimensional object. The object descriptor includes a view descriptor of the object as it appears 
in the image (one two-dimensional perspective view) and view descriptors of the object in other 
two-dimensional perspective views (not appearing in the image or sequence of images). See for 
example, the different views of the 3-D cylinder object illustrated in Figure 2 of the 
Specification. A search is initiated by inputting a query, for example, in the form of an outline 
which is the outline of the shape of a two-dimensional projection of the specific three- 
dimensional object of interest to the searcher. An object descriptor for the query object is 
derived and then compared with the stored object descriptors. More specifically, the query 
descriptor is compared with each of the plurality of perspective view descriptors in the stored 
object descriptor. If the query descriptor matches any of the perspective view descriptors, the 
overall object descriptor is considered a match and the corresponding image or sequence of 
images is retrieved, even if the perspective view of the object in the query does not match the 
perspective view of the object in the image or sequence of images. 

Independent claim 6 defines a method of searching for a query object in an image or 
sequence of images by processing signals corresponding to the images using a processor. 
(Specification p. 1, lines 4-5 and Fig. 1) The method includes, inter alia, providing a plurality of 
stored image representations of three-dimensional objects, each image representation being 
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associated with an object descriptor, each object descriptor including a plurality of view 
descriptors, each view descriptor representing the outline of the shape of a projection of each of 
the three-dimensional objects from a different perspective view of the three-dimensional object; 
(Specification p. 4, line 8 to p. 5, line 17, Figs. 1-3) inputting a query in the form of at least a 
two-dimensional outline of an object; (Specification page 6, lines 11-16 and lines 19-22, and Fig. 
5) deriving a query object descriptor of the query object; (Specification page 6, lines 17-18 and 
Fig.5) comparing said query object descriptor with at least one of said object descriptors; 
(Specification page 7, lines 3-21, page 8, lines 1-5, page 8, line 21 to page 9, line 11, and Figs. 3 
and 5) selecting and displaying at least one result corresponding to one of the image 
representations containing an object for which comparison between the associated object 
descriptor and the query object descriptor indicates a degree of similarity between the query 
object and said object. (Fig. 5, Specification page 9, lines 12-19). 

VI. GROUNDS OF REJECTION TO BE REVIEWED ON APPEAL 

A. Claims 6-10 and 33-41 stand rejected under 35 U.S.C. § 103(a) as being unpatentable 
over Eakins et al., "Retrieval of trade mark images by shape feature-the ARTISAN project), May 
22, 1996 ("Eakins") in view of U.S. Patent No. 6,801,641 B2 to Eraslan ("Eraslan"), and further 
in view of Japanese Publication No. JP 06-215105 to Haruo et al. ("Haruo"). 

VII. ARGUMENT 
Claims 6-10 and 33-41 

The rejection of claims 6-10 and 33-41 as unpatentable over the combination of Eakins, 
Eraslan and Haruo is improper because the combination fails to disclose "a plurality of stored 
image representations of three-dimensional objects, each image representation being associated 
with an object descriptor, each object descriptor including a plurality of view descriptors, each 
view descriptor representing the outline of the shape of a projection of each of the three- 
dimensional objects from a different perspective view of the three-dimensional object" as recited 
in claims 6 and 33, from which claims 7-10 and 34-41 depend. 

As discussed in the Specification, various methods, e.g., chain coding, quad-tree, and 
curvature scale space representations, of representing shapes of objects in images are known in 
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the art. In addition, methods of searching images using stored representations are known. 
However, these known methods have disadvantages. In most cases, an object appearing in an 
image or video sequence is a projection of a 3-dimensional real object onto a 2-dimensional 
plane. The 2-dimensional shape or outline of an object appearing in an image depends on factors 
such as the viewing position, viewing angle, and camera and optical system parameters. Objects 
will therefore have different outlines associated with different views. (See 3-D cylinder object 
illustrated in Fig. 2 of the Specification) However, the known methods of representing images 
treat these different outlines as different objects. As a result, if, for example, a user inputs a 
query based on a front view of an object A, and object A only appears in a video sequence from 
the rear and side view, known methods will not successfully match the query view of object A 
with the stored images including object A from the different view. The present invention 
overcomes this disadvantage with known methods by providing a system which associates 
different perspective views of a three-dimensional object appearing in an image or sequence of 
images. 

Accordingly, independent claim 6 defines a method of searching for a query 
object in an image or sequence of images by processing signals corresponding to the 
images using a processor. The method includes, inter alia, providing a plurality of stored 
image representations of three-dimensional objects, each image representation being 
associated with an object descriptor, each object descriptor including a plurality of view 
descriptors, each view descriptor representing the outline of the shape of a projection of 
each of the three-dimensional objects from a different perspective view of the three- 
dimensional object; inputting a query in the form of at least a two-dimensional outline of 
an object; deriving a query object descriptor of the query object; comparing said query 
object descriptor with at least one of said object descriptors; selecting and displaying at 
least one result corresponding to one of the image representations containing an object 
for which comparison between the associated object descriptor and the query object 
descriptor indicates a degree of similarity between the query object and said object. 

In rejecting claim 6, the Examiner asserts that "it would have been obvious to one [of] 
ordinary skill in the art at the time of the invention was made to modify Eakins' system to 
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include the different view of three-dimensional stored object in the database as taught by Eraslan 
in order to provide a faster search and retrieval system." In addition, the Examiner asserts that 
"Haruo also discloses providing a plurality of stored image representations of tree-dimensional 
[sic] objects, each image representation being associated with an object descriptor, each object 
descriptor including a plurality [of] view descriptors, each view descriptor of representing the 
outline of the shape of a projection of each one of the three-dimensional objects from a different 
perspective view of the three-dimensional object." Therefore, the Examiner concludes that 
because Haruo allegedly suggests an object having different shapes, it would have been obvious 
to "modify Eakins and Eraslan to include each view descriptor representing a shape of an object 
as disclosed in Haruo in order to allow the retrieval of an image using shape." The Examiner's 
assertions are unfounded for the following reasons. 

First, the Examiner's conclusion of obviousness is based on an incorrect reading of 
Eraslan. On page 8 of the Office action dated July 10, 2008 ("Action"), the Examiner asserts that 
"Eraslan teaches providing a plurality of stored image representations of tree-dimensional [sic] 
objects, each image representation being associated with an object descriptor, each object 
descriptor including a plurality [of] view descriptors, each view descriptor of [sic] representing 
the outline of the shape of a projection of each one of the three-dimensional objects from a 
different perspective view of the three-dimensional [object]." To support this assertion, the 
Examiner points to column 4, lines 2-8 and Fig. 8 of Eraslan. 

Although Fig. 8 of Eraslan illustrates different perspective views of a nose, Eraslan fails 
to disclose that each of these different perspective views of the nose attribute are stored as part of 
the object descriptor for the nose attribute or that a perspective view represents the outline of the 
shape of a two-dimensional projection of the nose from a different perspective. To the contrary, 
Fig. 8 of Eraslan illustrates the fact that with the face-attribute index (e.g., 01) and the specific 
face-feature surface code (142), a complete three-dimensional color face-feature surface which 
can be rendered graphically in the frontal view, left-profile view, right-profile view and all 
possible angled views of observation and analysis can be complied. See column 4, lines 35-60 
and Figs 7-13. Although Eraslan discloses that these 3-D surface maps can be used to analyze 
two-dimensional mug shots, nowhere in Eraslan is there any disclosure or suggestion of indexing 
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outlines of the shape of a two-dimensional projection of the three dimensional facial feature as 
claimed. Therefore, even if, arguendo, one where to equate the shape codes of Eraslan as being 
equivalent to the claimed view descriptors, the shape codes of Eraslan do not represent different 
outlines of the shape of a two-dimensional projection of the perspective view of the same 3-D 
object, but rather entire 3-D surface maps for different objects in the facial feature family. 
Accordingly, the Examiner's assertion that Eraslan provides object descriptors as claimed is 
unfounded. 

Second, the Examiner's assertion that Haruo also discloses providing object descriptors 
as claimed in incorrect. Although Haruo discloses a method for creating two-dimensional 
projections from three-dimensional geometric shape information, nowhere in Haruo is there any 
disclosure or suggestion of storing perspective view descriptors representing the outline of the 
shape of two-dimensional projection of a three-dimensional object in an image or sequence of 
images as claimed. At best, Haruo merely discloses a system for creating projections, not for 
creating view descriptors representing the outline of the shape of a two-dimensional projection as 
claimed. Therefore, even if one skilled in the art were motivated to combine Eraslan and Haruo, 
the combination would merely provide a system for creating geometric shape information for 
numerous facial features and a system for creating two-dimensional projections of the three- 
dimensional facial surfaces once constructed. However, the combination would still fail to teach 
providing perspective view descriptors as claimed. 

Finally, although Eakins discloses a system for searching for and retrieving similar 
images based in part on similarity between shape descriptors. The shape descriptors of Eakins 
are not equivalent to the claimed object or view descriptors. First, the shape descriptors of 
Eakins are not representative of three-dimensional objects within the stored images. To the 
contrary, the shape descriptors of Eakins are representative of an entire image, not a two- 
dimensional projection of a three-dimensional object as claimed. Second, the shape descriptors 
of Eakins are based on individual boundaries or families of boundaries extracted from a stored 
image (i.e., the boundary exists in the image), not different perspective views of a three- 
dimensional object which are not in the image. 
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Since Eakins, Eraslan, and Haruo each fail to disclose or suggest a method of searching 
for a query object in an image or sequence of images that includes, inter alia, providing a 
plurality of stored image representations of three-dimensional objects, each image representation 
being associated with an object descriptor, each object descriptor including a plurality of view 
descriptors, each view descriptor representing the outline of the shape of a projection of each of 
the three-dimensional objects from a different perspective view of the three-dimensional object 
as claimed, the combination of these three references cannot possibly disclose or suggest said 
feature. Therefore, even if one skilled in the art had some rationale to combine Eakins, Eraslan, 
and Haruo, which Appellant does not concede, the combination would still fail to render claim 6 
unpatentable because the combination fails to disclose each and every claimed element. 

Claims 7-10 and 33-41 are patentable over the combination of Eakins, Eraslan, and 
Haruo for at least those reasons presented above with respect to claim 6. 

VIII. CLAIMS 

A copy of the claims involved in the present appeal is attached hereto as Appendix A. 
The claims in Appendix A do not include the amendments filed by Applicant on January 9, 2009. 

Dated: July 8, 2009 Respectfully submitted, 




ichael K. Mujter 
Registration No.: 29,680 

BIRCH, STEWART, KOLASCH & BIRCH, LLP 

81 10 Gatehouse Road 

Suite 100 East P.O. Box 747 

Falls Church, Virginia 22040-0747 

(703) 205-8000 
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APPENDIX A 

Claims Involved in the Appeal of Application Serial No. 09/559,415 

6. (Previously Presented) A method of searching for an object in still or video images by 
processing signals corresponding to the images, the method comprising: 

providing a plurality of stored image representations of three-dimensional objects, each 
image representation being associated with an object descriptor, each object descriptor including 
a plurality of view descriptors, each view descriptor representing the outline of the shape of a 
projection of each of the three-dimensional objects from a different perspective view of the 
three-dimensional object; 

inputting a query in the form of at least a two-dimensional outline of an object; 

deriving a query object descriptor of the query object; 

comparing said query object descriptor with at least one of said object descriptors; 

selecting and displaying at least one result corresponding to one of the image 
representations containing an object for which comparison between the associated object 
descriptor and the query object descriptor indicates a degree of similarity between the query 
object and said object. 

7. (Previously Presented) A method as claimed in claim 6 wherein a query is input in the 
form of two or more two-dimensional outlines of an object, and wherein a query view descriptor 
is derived for each said outline, and wherein the step of comparing comprises comparing each 
said query view descriptor with each view descriptor in each stored object descriptor to derive a 
plurality of view-similarity values. 

8. (Previously Presented) A method as claimed in claim 7 wherein the view-similarity 
values are analyzed to derive object similarity values. 



9. (Previously Presented) A method as claimed in claim 6, wherein at least some of the 
object descriptors include view-independent descriptors which are related to shape and/or size of 
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the object, and wherein the method comprises inputting a view-independent query value and the 
step of comparing compares the query value with the view-independent descriptors for the stored 
object descriptors. 

10. (Previously Presented) A method as claimed in claim 6, wherein the query descriptor 
is derived using a curvature scale space representation of a query object outline. 

33. (Previously Presented) A method of searching for an object in still or video images by 
processing signals corresponding to images, the method comprising: 

receiving a query descriptor representing a two-dimensional view of a query object; 

comparing the query descriptor with a plurality of stored object descriptors each 
representing a three-dimensional object, each object descriptor including a plurality of view 
descriptors, each view descriptor representing the outline of the shape of a projection of each of 
the three-dimensional objects from a different perspective view of the three-dimensional object, 
each stored object descriptor being associated with an image representation of the object; and 

selecting the three-dimensional object and associated image representation when of the 
respective stored object descriptors matches the query descriptor. 

34. (Previously Presented) The method of searching according to claim 33, wherein the 
query descriptor is derived using a curvature scale space representation of an outline of the query 
object. 

35. (Previously Presented) The method of searching according to claim 34, wherein the 
stored descriptor is derived using a curvature scale space representation of an outline of the 
three-dimensional object. 

36. (Previously Presented) The method of claim 6, further comprising: 
deriving an object descriptor for an object in an image by: 

deriving a view descriptor of a first outline of a three-dimensional object in the image, 
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deriving at least one additional view descriptor of the outline of the object in a different 
perspective view from the perspective view in the image, and 

associating the two or more view descriptors to form the object descriptor. 

37. (Previously Presented) The method of claim 6, wherein said selecting and displaying 
includes selecting and displaying an image representation of an object having a different 
perspective view from perspective view of said query object based on said query object matching 
with at least two view descriptors including a view descriptor not representing perspective view 
of the object in the image representation. 

38. (Previously Presented) The method of claim 33 wherein one of the view descriptors 
corresponds to a view of the object as the object appears in the respective image representation. 

39. (Previously Presented) The method of claim 33 wherein one of the view descriptors 
corresponds to a perspective view of the object different from the perspective view of the object 
as the object appears in the respective image. 

40. (Previously Presented) The method of claim 33, wherein said selecting includes 
selecting and displaying an image representation including an object having a different 
perspective view from perspective view of said query object based on said query object matching 
with at least two view descriptors including a view descriptor not representing view of the object 
in the image. 

41. (Previously Presented) The method of claim 6, wherein each said view descriptor is a 
different representation of the object from a different perspective view of the three-dimensional 
object. 
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APPENDIX B 

No evidence pursuant to §§ 1.130, 1.131, or 1.132 or entered by or relied upon by the 
examiner is being submitted. 
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APPENDIX C 

No related proceedings are referenced in II. above, hence copies of decisions in related 
proceedings are not provided. 
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